Document Embeddings via Recurrent Language Models

Document embeddings serve to supply richer semantic content for downstream tasks which require fixed length inputs. We propose a novel unsupervised framework by which to train document vectors by using a modified Recurrent Neural Network Language Model, which we call DRNNLM, incorporating a document vector into the calculation of the hidden state and prediction at each time step. Our goal is to show that this framework can effectively train document vectors to encapsulate semantic content and be used for downstream document classification tasks.